Search AI Products and News

On April 7th, Kimi Open Platform officially announced price adjustments for its model inference services, significantly lowering context cache prices as well. This is based on a year's worth of technological advancements and performance optimizations by Moonshot AI. This move signifies Kimi Open Platform's commitment to enhancing user experience and promoting the widespread adoption of AI services while improving its technical capabilities. According to Kimi Open Platform, these price reductions are a result of Moonshot AI's progress in model training and inference optimization over the past year.

Researchers from DeepSeek and Tsinghua University recently published a new paper exploring scaling methods for reward model inference, seemingly advancing DeepSeek R2. Reinforcement learning is widely used in the large-scale post-training phase of large language models, but faces the challenge of obtaining accurate reward signals for these models. The researchers found that using pointwise generative reward modeling (GRM) improves model adaptability and scalability during inference. To this end, they propose Self-Principle Calibration Tuning (SPCT) learning.

Recently, Professor Zhai Jidong's team from the Institute of High Performance Computing at Tsinghua University, in collaboration with Tsinghua-affiliated innovative enterprise, Qingcheng Extreme Intelligence, announced the open-sourcing of Chitu, a groundbreaking large model inference engine. This innovative technology marks another significant breakthrough for China in the AI field, particularly in inference engine development. A core highlight of the Chitu engine is its ability to natively run FP8 precision models on non-Nvidia Hopper architecture GPUs and various domestic chips.

New York-based AI team Nous Research, known for its personalized, unrestricted language models, has launched a new model inference API. This marks a significant step for the unconventional AI group, making its advanced language models accessible to a wider range of developers and researchers through a programmatic interface. The unrestricted nature of these models has always set Nous Research apart, challenging the likes of OpenAI and Anthropic...

The ByteDance Doubao large model team announced today the successful development of a new sparse model architecture called UltraMem. This architecture effectively addresses the high memory access issues during the inference of MoE (Mixture of Experts) models, improving inference speed by 2 to 6 times compared to MoE, and reducing inference costs by up to 83%. This groundbreaking advancement opens a new path for efficient inference of large models. The UltraMem architecture successfully resolves the memory bottleneck during inference of MoE architectures while maintaining model performance. Experimental results show that the parameters and activation conditions are the same.

Generative language models face many challenges during the transition from training to practical applications. One major issue is how to achieve optimal performance during the inference stage. Current strategies, such as Reinforcement Learning from Human Feedback (RLHF), primarily focus on improving the model's win rate but often overlook decoding strategies during inference, such as Best-of-N sampling and controlled decoding. This gap between training objectives and actual usage may lead to inefficiencies and affect the quality and reliability of outputs. To address these issues, Google D

Kimi Technology Co., Ltd. and Tsinghua University's MADSys laboratory have jointly released an open-source project called Mooncake, aimed at collaboratively building a large model inference architecture centered around KVCache. In June 2024, both parties released the design plan for the Mooncake inference system based on Kimi, which significantly enhances inference throughput through PD separation and a storage-computation architecture, garnering widespread attention in the industry.

In an era of rapid development in artificial intelligence, the intelligence level of large models continues to improve, but the challenges of inference system efficiency are becoming increasingly apparent. Addressing high inference loads, reducing inference costs, and shortening response times has become an important issue faced by the industry. Kimi has collaborated with the MADSys laboratory at Tsinghua University to launch the KVCache-based Mooncake inference system design plan, which will be officially released in June 2024. The Mooncake inference system enhances efficiency through its innovative approach.

In the world of artificial intelligence, the reasoning capabilities of machine learning models, particularly large language models (LLMs), have been a focal point for scientists. Recently, Apple's AI research team published a paper titled 'Understanding the Limitations of Large Language Models in Mathematical Reasoning,' shedding light on these models' limitations when addressing logical problems. In the paper, researchers demonstrate this through a simple mathematical question. They first posed a question about Oliver picking kiwis: as follows: Oliver picked 44 on Friday.

Recently, a project called Q * was released by the team led by Yan Shuicheng from Kunlun Wanwei in China, in collaboration with a research team from Nanyang Technological University in Singapore. The project aims to improve the inference capabilities of small models. This project differs from OpenAI, yet allows small models to achieve inference abilities comparable to models with tens or even hundreds of times more parameters.

Recently, Kunlun Wanwei, in collaboration with Nanyang Technological University in Singapore, successfully developed an algorithm called Q* that significantly enhances the inference capabilities of existing large models. Q* enables smaller models to achieve inference capabilities comparable to those of models with tens or even hundreds of times more parameters across various datasets, leading to a substantial improvement in model performance while dramatically reducing the demand for computational resources. This brings new possibilities for the widespread application of artificial intelligence and ushers in a new era of efficient intelligence.

The inference cost of Baidu Wenxin large model has been reduced to 1% of the original. The daily call volume of the Wenxin large model exceeds 50 million, with a quarterly growth rate of 190%. Approximately 26,000 enterprises are using the Wenxin large model, with a quarterly growth rate of 150%.

Deep learning researchers have found that the performance of language models in logical reasoning remains a significant challenge. Recent research reveals that the order of premises in tasks has a significant impact on the logical reasoning performance of language models. The findings may guide experts in their decisions when using language models for fundamental reasoning tasks.

Intel's Gaudi2 technology matches NVIDIA's AI accelerators in large-scale language model inference. Gaudi2 shows better inference performance than the NVIDIA A100, with higher memory bandwidth utilization. The cost-performance ratio for both training and inference exceeds that of the NVIDIA A100 and H100. New data confirms Intel's performance in large-scale language model inference. Gaudi3 technology is expected to be released in 2024, bringing significant performance advancements.

Research shows that Intel's Gaudi2 technology competes with NVIDIA's AI accelerators in large-scale language model inference. The inference performance of Gaudi2 is comparable to NVIDIA's H100 system in decoding and surpasses that of the A100. According to public cloud pricing, Gaudi2 offers better value for training and inference compared to NVIDIA's A100 and H100. Intel's Gaudi3 is planned for release in 2024, promising four times the processing power and double the network bandwidth. Intel is committed to advancing high-performance computing and

Trends in the Application of Large Models: The article mentions how large model AI is rapidly infiltrating various industries, but at the same time exposes the logical reasoning flaws of large models. Limitations of Knowledge in Large Models: A founding member of OpenAI candidly stated that the knowledge of large models is much more incomplete than you might imagine. Tests in both fictional and real scenarios indicate that large models suffer from a 'Reversal Curse' and are unable to perform reverse reasoning. Doubts About the Reliability of Large Models: Logical reasoning forms the basis for large models to solve problems correctly. The Reversal Curse indicates deficiencies in the reasoning capabilities of large models, which may impact various important application areas.